[stormshield] Add thematic dashboards and rework the Overview#20088
Draft
arenard wants to merge 4 commits into
Draft
[stormshield] Add thematic dashboards and rework the Overview#20088arenard wants to merge 4 commits into
arenard wants to merge 4 commits into
Conversation
Fix the copy-pasted descriptions of the default and count ingest pipelines, remove field definitions that never appear in events (the pipeline consumes tz into event.timezone, converts duration into event.duration, renames msg to message, and drops origdst/origdstport after converting them to destination fields), and correct the documentation: the agent syslog processor is configured for RFC5424, so the README no longer presents the Legacy format as supported, and the compatibility section now covers SNS 4.x and 5.x, which share the same WELF key-value audit log format. Also document how to collect SNS IPFIX flow records with the NetFlow integration, update vendor links to the SNS v5 documentation, and use arrows for menu paths per the documentation style guide.
a2b2382 to
566d6e0
Compare
Contributor
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
1b6796f to
06384f6
Compare
Contributor
🚀 Benchmarks reportTo see the full report comment with |
Set event.kind, event.category and event.type for every SNS log family through a table-driven Painless script keyed on logtype, with allowed/denied event.type modifiers derived case-insensitively from the rule action. The periodic statistics families (monitor, filterstat, count, routerstat, authstat, ipsecstat) are tagged event.kind: metric so health telemetry can be excluded from detection scopes with a single filter. Unknown families fall back to event/network/info, covered by a dedicated test sample. The script only runs on documents without an error, so the pipeline_error kind set by sub-pipeline failure handlers is preserved. Add the ECS network mappings tracked in elastic#10114: rcvd to destination.bytes (mirroring the existing sent mapping), network.bytes as the sum of both, srcif/srcifname/dstif/dstifname copied to observer.ingress and observer.egress interface fields, and network.direction computed with private ranges as internal networks. Promote alarm fields (alarmid to event.code, risk to event.risk_score, pri to event.severity on the alarm scale where 1 is major and 4 is minor), HTTP proxy fields (arg to url.original, url.domain from destination.domain), and populate related.hosts. All mappings are copies; no stormshield.* field moves or changes location. Guard the ASN geoip processors with null checks, matching the geo processors. Fix the count sub-pipeline, which failed on every count log since creation: the script called splitByToken, which does not exist in Painless, and collapsed all rules into a single map. Rule counters are now a sorted array under stormshield.metadata.rule_stats. Extend the metadata move list with the documented alarm, proxy, vpn and auth fields so they land under stormshield.metadata instead of being left unmapped. Add pipeline test samples for the alarm, auth, web, vpn, xvpn, count and filter families, derived from the log format examples in the official Stormshield documentation with sanitized addresses, including uppercase and empty action values. Add an Alarms (IPS) dashboard keyed on the new categorization fields. Document the categorization table in the README.
…metadata allow-list Derive `event.outcome` on the auth, server and xvpn families from the vendor `error` return code (`0` maps to success, other codes to failure). The official field documentation lists exactly these three families for `error`; on vpn logs the same key is a severity level and is left untouched. The vendor documentation pairs `0`-first numeric examples with `Success`-first display labels without stating the convention outright, so the mapping follows the strong implication of the field tables. Map `domain` to `user.domain` and `usergroup` to `user.group.name`, splitting the comma-separated multi-valued form that SNS 5.x emits. On web logs, map `op` to `http.request.method` and `result` to `http.response.status_code`; on other families these keys carry protocol-specific codes (FTP, Modbus) and are not promoted. Map the module name in `service` to `process.name` on the system, sandboxing and routing families only — on pvm logs the same key names the vulnerable product detected on a scanned host. Set `observer.product`. All mappings are copies; the vendor keys remain available under `stormshield.metadata`. Extend the `stormshield.metadata` allow-list with the documented vendor keys that previously leaked as undeclared `stormshield.*` fields: host and IP reputation (`srchostrep`, `dsthostrep`, `srciprep`, `dstiprep`), alarm packet captures (`pktlen`, `pktdump`, `pktdumplen`), Ethernet protocol (`etherproto`), SSO agent (`agentid`), IKE peer name (`peername`), and the industrial-protocol plugin fields (`unitid`/`unit_id`, `cipservicecode`, `cipclassid`, `requestmode`, `responsemode`, `UI`, `error_class`, `error_code`, `format`). The NAT reading of the pipeline was verified against the official field tables while preparing this change: `origdst`/`origdstport` are documented as the destination before translation and map to `destination.ip`/`destination.port`; `dst` is the translated destination and maps to `destination.nat.ip` when it differs. A connection sample with the documented example values now locks this in the pipeline tests, alongside new samples for the alarm reputation and packet-capture fields, a Modbus plugin line (whose `result` stays under metadata and does not produce `http.*`), a server line, a system line, and multi- and single-valued `usergroup` forms.
Add three thematic dashboards keyed on the log families the pipeline now categorizes: Network Traffic (connection/filter events, verdicts, endpoints, rules, byte volumes), Web Activity (domains, HTTP methods and return codes, site categories, volumes) and Users & VPN (authentication, administration sessions and IPsec/SSL VPN events with their outcomes, users, directories, groups and peers). Every dashboard scopes its data with dashboard-level `data_stream.dataset` and `stormshield.logtype` filters, keeps panel queries empty, and drops the log-type control that its own filter would contradict. Add a links panel at the top of the Overview pointing to the four other dashboards, extend it with an appliance health row covering the periodic statistics families (scoped with panel-level queries, since the Overview carries no log-type filter), title its event-count metric panel, and replace the terms aggregation on `event.duration` with a median over time. Fix the dashboard controls, which carried no data view reference and were silently dropped by Kibana at render time. Rename all dashboards to the `[Stormshield SNS]` title prefix so future Stormshield product integrations can share the vendor namespace unambiguously. Replace the outdated Overview screenshot with captures of the five dashboards rendered on a live stack.
06384f6 to
5568233
Compare
Contributor
|
Changelog link mismatch — expected
Tip If expected, add the |
💔 Build Failed
Failed CI StepsHistory
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed commit message
Add three thematic dashboards keyed on the log families the pipeline now categorizes: Network Traffic (connection/filter events, verdicts, endpoints, rules, byte volumes), Web Activity (domains, HTTP methods and return codes, site categories, volumes) and Users & VPN (authentication, administration sessions and IPsec/SSL VPN events with their outcomes, users, directories, groups and peers). Every dashboard scopes its data with dashboard-level
data_stream.datasetandstormshield.logtypefilters, keeps panel queries empty, and drops the log-type control that its own filter would contradict.Add a links panel at the top of the Overview pointing to the four other dashboards, extend it with an appliance health row covering the periodic statistics families (scoped with panel-level queries, since the Overview carries no log-type filter), title its event-count metric panel, and replace the terms aggregation on
event.durationwith a median over time. Fix the dashboard controls, which carried no data view reference and were silently dropped by Kibana at render time. Rename all dashboards to the[Stormshield SNS]title prefix so future Stormshield product integrations can share the vendor namespace unambiguously. Replace the outdated Overview screenshot with captures of the five dashboards rendered on a live stack.Design notes
^8.19.0 || ^9.1.0, while this package supports^8.11.4. The appliance health row of the Overview is the natural candidate for a collapsed section once the version floor moves.Checklist
changelog.ymlfile.Author's Checklist
elastic-package checkpasses and the package installs into Kibana 9.4.3: all five dashboards import, the Overview resolves its four dashboard links, and rendering shows no dropped-panel warningsHow to test this PR locally
Related issues